Design of computerized system contributed to the death of ten USN sailors

On 21 August 2017 the US Navy destroyer John S McCain collided with a civilian tanker near Singapore, resulting in the deaths of ten US sailors, and $100M in damage. The US National Transportation Safety Board report found the probable cause of the collision was "... a lack of effective operational oversight of the destroyer by the US Navy ...". However, also contributing to the accident was the computerized steering system: "Also contributing to the accident was the operation of the steering system in backup manual mode, which allowed for an unintentional, unilateral transfer of steering control.". This would be a useful report for students of safety critical systems to study.

John S McCain Bridge Control Station.
Drawing from IBNS technical manual;
color added by NTSB. Figure 4 of
NTSB/MAR-19/01 PB2019-100970.

The destroyer was equipped with bridge control stations with flat-panel touch screens, and a graphical user interface (GUI), in additional to a conventional steering wheel. The stations had an "emergency override to manual" function activated by what the crew referred to as the "big red button". This was intended to provide manual control in the event of a computer malfunction, and the designers no doubt thought it was foolproof: press the red button and steer the ship with the wheel.

However, as the NTSB detailed, the crew unintentionally
transferred control of steering from one station to another, but interpreted this as a failure of steering. This confusion may have been because the crew were uncomfortable with the automated mode of the system, and preferred to use the backup manual mode. However the backup mode was not intended for normal use, and allowed the control to be transferred without the operator noticing.

The  NTSB recommended crew being instructed to only operate the system in manual mode during an emergency. However, the underlying problem appears to be that the crew did not trust the automated system. This would require training the crew so they felt they could rely on the system, or to redesign the system to provide more intuitive feedback. Part of the intuitive feedback, I suggest, could be via manual controls.

The Bridge Control Stations have a ship's wheel, but this is an input only device, and there are no physical engine throttles, just a GUI display. One way aircraft cockpit interfaces display the operation of the autopilot is by physically moving the throttle levers. Similarly, in aircraft with a control yoke  the control moves in response to auto-pilot commands, and also provides tactile and visual feedback of the control responses put in by the co-pilot. The autopilot can be overridden simply by moving the yoke. If implemented on the ships bridge control, this would provide intuitive feedback as to who is in control, and an intuitive way to take control. The operator would be able to see, and feel, inputs through the wheel and throttles. If they wanted to override the automated system, or another operator, they just need to move the controls.

Some modern aircraft lack the visual and tactile feedback in controls, particularly those using side-stick controllers, rather than a yoke. However, in aircraft pilots receive intensive training in the use of these systems, and are sitting close to each other in the cockpit, so are usually able to see and hear what each other are doing. Even so, conflicting inputs have lead to aircraft accidents.  On a ship the operators are much further apart, which makes coordination much more difficult.

Reference

Collision between US Navy Destroyer John S McCain and Tanker Alnic MC Singapore Strait, 5 Miles Northeast of Horsburgh Lighthouse
August 21, 2017, Marine Accident Report, National Transportation
Safety Board, NTSB/MAR-19/01 PB2019-100970, Notation 58325
Adopted June 19, 2019 URL https://assets.documentcloud.org/documents/6243999/MAR1901.pdf

FastMail Classic interface has now been retired

FastMail have discontinued the "Classic" web interface for their e-mail service. In place of the fast web pages, I have to wait for the slow "responsive" interface. This is most noticeable when using a slow wireless modem.

Build your Own $10 Virtual Reality Glasses

Marcus Hutter and Stephen Gould will speak on "Build your Own 3D Virtual Reality Glasses for 10$" at the Australian National University in Canberra, 4pm, 13 February 2014.The DIY 3D Virtual Reality Goggles are built using a smart phone and part of a Student Project.
It occurs to me that this is something which should work well with one of the new smart phones with curved display. The case to hold the phone could be manufactured with a 3D printer (with the design by the engineering students), the project crowd-funded and entered in the Innovation ACT start-up competition.

ABSTRACT:
3D television and 3D cinema are becoming wide-spread, but 3D Virtual Reality goggles (or glasses), despite being a many decades old idea, are just receiving increased attention. The few commercial devices available don't sell well, in parts due to price, but a cheap alternative has emerged: A modern SmartPhone, two lenses for under 10$ and some cardboard and glue is all you need to build your own 3D-VR goggles. 

Marcus will bring his to the seminar for you to try out, and will share his experience in building and using them. Stephen will discuss potential computer vision applications and student projects. ...

Mobile Web Interface Better Bet on Desktop Computer

One of my house-guests recently wanted to place a bet on-line with tab.com.au using my desktop computer (which has the Firefox for Linux browser). The site reported "You need to have the latest version of Adobe Flash Player installed to access the wealth of features ...". But it also said "You can REGISTER, DEPOSIT and PLACE BETS using tab.mobi if you can't download or update Flash.". This simplified mobile web interface looked a little odd on a 24 Inch monitor, with a small amount of very large text. But my friend commented it was faster and easier to use that the Flash interface they usually used. It does make me wonder why interface designers produce complex "flash" designs, when the punter just wants to quickly place a bet.

Prototyping Combat Workstations for Submarines

Greetings from the Information & Human Centred Computing Group at the Australian National University in Canberra, where Bruce Thomas, from University of South Australia is speaking. on building prototype workstations to try put the interface for systems, such as the combat system on a submarine. Using the standard techniques applied in interface design a non functioning "mock-up of the interface would be first built, then a prototype with real controls. The mock-up is very flexible but cannot be used for more than very early tests. The prototype can be very close to the look and operation of a production device, but cannot be easily changed. Bruce is looking at using sensors which can detect the position of the operator's fingers, with simulations of controls which can be changed quickly. As interesting aspect of this is that the interfaces for real devices, such as combat systems are using the sort of interfaces Bruce is designing for prototypes. Military consoles are tending replace special purpose switches and buttons with a touch screen, a general purpose pointing device.

While Austrlaia does not design a new submarine very day, there are many other military systems which need interfaces to electronic systems. The pratice has been to simply add a new control panel for each new items of equipment. However, the physical space in aircraft, ships and vehicles, and the capacity of the operator to uses the assorted controls is limited. 

Having Robots Understand Hand Signals

Computer scientists are training robots to communicate with humans trough other than speech, keyboards, and mice. One useful approach is to mimicking signals already used between people and with animals. As an example, the construction industry uses hand signals for communication with machinery operators, such as Tower Crane Hand Signals and robot aircraft are being programmed to follow Aircraft Marshalling Hand Signals.

Designing a better user interface for bomb disposal

The documentary TV series "Bomb Patrol Afghanistan" shows a US Navy Explosive Ordnance Disposal (EOD) unit working in Afghanistan. In the first episode "Rules of Engagement", the team is shown operating a remote control robot from their armoured truck. The the team leader sits in the front of the truck next to the driver, where they have a good view of the area and the progress of the robot. However, the robot operator sits in the back of the truck where they cannot see out. This causes problems for the team.

The robot operator relies on a small  display in front of one eye, which shows then a view from a camera mounted on the robot. The operator concentrates intensely, using a video game controller to direct the robot. At the same time the team leader, who can see the robot, but not what the operator is doing behind them, asks for an update "tell me what you are doing". The operator is distracted and the robot tips over into a crater. The operator is able to right the robot and drive out of the crater, but clearly the computer interface for the robot and the communication between the crew could be improved.

Normally in an armoured fighting vehicle (AFV), such as a tank, the commander sits towards the rear. The crew sit in front and beside the commander. The commander and crew in the turret share a common viewpoint. The other crew know the general direction of interest by where the turret is pointing, even if they cannot see out. In some cases the electronic sight's view is relayed to multiple crew stations.

In contrast to an AFV,  the commander in a Mine-Resistant Ambush Protected Vehicle (MRAP), normally sits at the front next to the driver. The commander has a good view of the outside and can communicate easily with the driver. But the commander can't see or easily communicate with the personnel behind them in the vehicle. This is not so important when the MRAP is used for its origial role, essentially an armoured truck to transport personnel safely to a location where they dismount to work. However, if the personnel have to operate from within the vehicle, then communication is a problem.

Ironically the viewers of the Bomb Patrol documentary had a much better view outside than the crew in the back of the truck in the documentary. The documentary makers had equipped the vehicle with numerous cameras, to shown the view outside, to show the crew and also a camera on the team leader's helmet, to show their viewpoint. This suggests a solution to the communications problem, without having to radically change the layout of the vehicle: cameras could show the crew what the team leader is looking at and show the team leader what the crew are doing.

Australia's Squeeze Toy Navy

One of the fun parts of a trade exhibition is picking up the giveaways. At the Pacific 2012 Maritime Defence Conference in Sydney, I collected five squeeze toy ships. These are, front to back: Collins Class submarine from the Australian Navy, unidentified submarine from L3, Anzac class frigate from Australian Navy, Canberra Class Landing Helicopter Dock Ship from BAE Systems, unidentified frigate from Australian Submarine Corporation.

Also on the stand of Australian company, Austral, was a Cars 2 Action Agents Battle Station Playset by Mattel, which is clearly modeled on their
Independence class littoral combat ship built for the US Navy (with out some of the expensive options it is the MRV 80).

Learning Amphibious Operations from Europe

Greetings from the Pacific 2012 Maritime Defence Conference in Sydney, where DSTO is hosting a stream on Technology for Amphibious Operations session. Interestingly the speakers are from Spain and the UK, as new ships have been purchased from these countries. This must be culture shock for the Australian Department of Defence and Australian Navy, who normally look to the US Navy and American industry.

It might seem a ship is just a ship and you can use it however you like. But a particular way of working is built into the design of the ships. Also the relatively small European defense forces are closer in the way they operate to Australia.

There are considerable challenges for Australia to adapt European experience, as the distances involved are much larger and they will need to learn to work with the US 7th Fleet (commanded from the USS Blue Ridge).

The need for bridging different approaches is best illustrated by the Canberra Class Ships. The main structure of the ship is being built in Spain and then will be transported to Australia. The "island" with the bridge of the ship, radars and other electronic systems, is being built in Australia. The island will then be placed on the ship and everyone will hope the two parts work together. In the same way the ship has to work with navy, army, air force and civilian personnel aboard and work with the US Navy.

Looking at some of the screen-shots of ICT systems used for planning amphibious operations on display at Pacific 2012, it struck me that these look very dated. The systems have a lot of text based forms and spreadsheet-like tables. These are typically presented using the X-Windows system on a graphical screen. Such a display can be hard to understand for new personnel, or those under stress (both of which are normal on an amphibious operation). The i-Pad like interface with big bold icons and a touch interface, as used on SAAB's 9LV SAT, might improve usability

Remote Assistance for Real World Tasks via the Internet

Greetings from HCC (and friends) at the Australian National Unviersity in Canberra, where Matt Adcock (CSIRO) is talking about Depth Sensing Cameras . CSIRO is research "remote assistance" where a worker is provided with assistacne over the internet from an expert at a remote location. One example is CSIRO’s ReMoTe (Remote Mobile Tele Assistance) for mine workers and RIDES (Remote Immersive Diagnostic Examination) for health workers. These systems rely on use of specialized sensors. More recent research uses the Microsoft Kinect games sensor and uses WebML for display, to provide a low cost system.

The Kinect comes with basic code for developers to detect the location of people and their limbs in a scene, but not positioning of fingers, as would be required for manual tasks. With additional software the Kinnect can be used to create a virtual multi-touch interface.

As well as maintenance and medical uses, this interface might be applicable to military uses. This would include interpretation of Army Hand and Arm Signals. As an example, a mule vehicle carrying supplies for a patrol could be controlled with the same hand signals as used for communication between personnel. Another use would be the control of complex operations, such as the Airbus Military Aerial Refuelling Boom System (ARBS) for Airbus A330 MRTT tanker aircraft, which has proved problematic with more conventional controls.

The technology also has potential for domestic use, enhancing video conferences and for scanning the inside of a room for virtual redecorating.

QANTAS Frequent Flyer Icons

Recently I was renewing my QANTAS Club membership (ACS members get a discount). I noticed that QANTAS were using frequent flyer icons and offering a Frequent Flyer points iGoogle Gadget. There are seven icons (more correctly pictograms).

I encourage my web design students to use pictograms, but care needs to be taken with their selection and use. In this case QANTAS has include a caption in each icon, which reduces confusion (but is a problem for multiple languages). The colour of each QANTAS icon indicates if the customer has earned points recently (red: points earned, grey: no points).The use of colour for status alone is not a good idea and does not meet accessibility guidelines. Also there can be confusion over the meaning of the icons. In this case, for example, green would be more appropriate to indicate that there has been recent activity (as in traffic lights: green for "go") and red for no activity (red: "stopped"). Use of grey is ambiguous, as it is not a colour.

1. Fly: For points earned flying. This icon resembles the "Departing Flights" pictogram of AIGA/U.S. Department of Transportation.
2. Shop: For points at Woolworths Group stores. The meaning of the shopping trolley is reasonably clear.
3. Pay: For payment by credit and charge cards. This symbol is not so clear and might be mistaken for a hamburger (fast food).
4. Stay: For hotel partners. This symbol is commonly used for accommodation.
5. Hire: For car rental.
6. Eat: For restaurants. Similar to the DOT "Restaurant" pictogram.
   
7. More: For other partners in the QANTAS scheme. This is the DOT pictogram for "First Aid". Its use here is confusing and inappropriate. A better symbol might be three dots.

Mozilla Raindrop message manager for Google Wave

Mozilla Raindrop message manager is intended to combine email, twitter, blogs and social networking in one conversation. In "Introducing Raindrop" (22 October 2009) the Raindrop Team (who developed Thunderbird) explain the Raindrop 0.1 prototype.

From the explanation, it seems to me that Raindrop might provide some of what I found missing from Google Wave. Having sat through the Google Wave Hackathon Day the problem I most had was with the user interface. The protocols and architecture behind Google Wave are elegant, but the user interface is not. Perhaps Raindrop could be a people-centric front end to Wave. After all raindrops can result in Waves. ;-)

There is a flash video explaining the Raindrop design. The first iteration of the interface design "Inflow" shows just two panels on the screen, with a conversation shown by intending of the text. The second iteration of the interface design is called an "Inflow Grid", with a rectangular table of messages.

In all of this it should be remembered that there has been considerable work by others on how to intelligently sort messages (such as by Eric McCreath and Judy Kay).

ACT Library Self Check Problems

The ACT Government has installed new Self Check Out machines in the ACT Library at Belconnen. While the machines work well mechanically, the printed instructions on the units do not match the actual operation. I asked the library staff member on duty and they explained that they were not permitted to change the instructions without permission from senior management. I suggest the senior management needs to visit the Belconnen Library, be briefed by the staff, authorise a change and have it done promptly. If the management can't do this, then Jon Stanhope, Minister for Territory and Municipal Services needs to replace the management.